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ABSTRACT 

A study to determine whether five Pitts]3urgh 
hospitals have communication training programs and whether the 
programs have been or are currently teing evaluated examined the 
following research questions: (1) whether they have ever used a 
communication training program (i.e., writing, interpersonal 
communication, puDlic speaking, group leadership); (2) if so, what 
was the background of the program (i.e., content, who initiated the 
program, where war it conducted, how long did it last); (3) why it 
was decided that the communication program was necessary; (4) how 
successful the program was (i.e., formal evaluation, based on 
intuition or comments heard t>y trainer, pencil and paper test); and 
(5) whether the communication training prbgreun was conducted by 
in-house personnel or by outside consultants. Subjects, training 
directors of the five hospitals, were interviewed and given a brief 
background of why the study was being conducted. Results demonstrated 
that communication training programs do exist and that the programs 
are evaluated in some way. However, in no cases that were discussed, 
were measurements used to est£a)lish baselines before and after 
training to determine how they affect the organization's overall 
goals. Therefore, it seems clear that much more investigation and 
research is needed to determine if baselines can be constructed, and 
if training programs can be accurately assessed using this type of 
evaluation. (Three tables of data are included, 10 references and 2 
appendixes are attached.) (RAE) 
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In 1987, U.S. organizations budgeted $32 billion for training 
staffs and programs (Lee, 1987). When over 2,400 of these 
organizations were asked what type of training thoy do most, the 
top three responses were management skills/development, 
supervisory skills, and communication skills (Lee, 1987). If we 
consider that even management skills/development and supervisory 
skills may contain elements of communication skills, it is 
apparent that most training in corporate America utilizes some 
form of communication training. At the sanie time that vast 
resources are spent on commimication training, there appears to be 
a general lack of effort and resources spent on evaluating these 
programs (Blakeslee, 1982, Campbell, 1975, Goldhaber, 1986, Laird, 
1985)* Sometimes no real evaluation is attempted and other times 
evaluation is based on intuition^r and not data (Campbell, 1975)* 
The problem with this type of evaluation is that it is often 
superficial and does not provide an accurate assessment of 
training programs. Therefore, knowledge of why a program succeeds 
or fails cannot be cjained. 

Certainly, the day-to-day demands of the work environment 
make it difficult to conduct thorough evaluation. Trainers may be 
responsible for teaching several subjects as well as have other 
organizational responsibilities. Kelly and Baird (1984) have 
suggested four organizational "realities" that make it difficult 
to evaluate training programs. First, some organizations may not 
want extensive quantitative or qualitative evaluation. Second, 
extensive evaluation procedures cost organizations in terms of 
time and dollars. Third, organizations often don't know what 



training is supposed to improve. Finally, extensive evaluation 
methods may be threatening to management. If a program flops, yet 
it had the complete support of management, someone is bound to 
loose face* 

Even though evaluation is difficult, many authors suggest 
that is should and must be attempted with almost every training 
prograon (Blakeslee, 1982, Campbell, 1985, Forster, 1977, 
Goldhaber, 1986, Komras, 1985, Laird, 1985, Mc^ehee, 1977). These 
authors suggest that evaluation is best when it is planned and 
implemented as part of the training progreun* However, the 
question rmiainsi what should be evaluated? Should the program 
content, trainer, trainee, or something else be assessed? Laird 
(1985) and Odiorne (1961) suggest that evaluation can occur on at 
least three levels: contribution to organizational goals, 
achievement of learning objectives, and/or perceptions of the 
trainees and their managers* 
Contribution to Organisational Goals s 

Some forms of training are conducted to help reduce or solve 
a problem within the organization. In other words, training is 
directed toward organizational urgencies* Evaluation of training 
conducted to determine whether the urgent problem was eliminated 
. or alleviated* To find out how well the training worked, some 
form of quantification is needed* Quantification will provide the 
trainer or management staff with a baseline* For example, the 
nuniber of units of work per hour, tasks completed, budgets 
submitted^ machine downtime, and so on could be used aa a 
baseline* Next, a decision would be made as to "how much" these 
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baselines needed to be changed. After these preliminary measures 
were decided upon, then training would be implemented* Naturally, 
after the training is completed, the new baseline is determined* 
If the new baseline level sufficiently exceeds the previous level, 
then the program would be called successful* The training, 
therefore, would have helped solve a problem* 
Achittvanittnt of Learning Objectives t 

Second, training can be evaluated based on how well 
participants achieved the learning objectives* It is not likely 
that all participants will achieve all o£ the learning objectives 
100% of the time* However, the more specific the learning 
objectives, the easier it is to test each participant and decide 
whether the objective was met* This form of evaluation usually 
occurs immediately after the training program in the form of a 
short quiz* More elaborate forms of evaluation use on the job 
assessments by either a supervisor or trainer* Interestingly 
enough, this form of evaluation can also be quantified using a 
four step model: 1) compute the potential (number of students 
multiplied by number of goals), 2) test individual achievements 
(test each student on each objective), 3) compute gross 
achievements (add all the ""yes** achievements), and 4) compute 
achievement quota (divide step 3 by step 1)* This formula allows 
the trainer to see what percentage of total learning objectives 
were achieved* 

Perceptions of Trainees and Their Hanagerst 

Finally # evaluation can be based on perceptions* This form 
of evaluation usually follows the opinion survey format, where 




participants can respond to scalar questions (e.g., always, often, 
now and then, seldom, never) or open questions (e.g., what course 
activities would you eliminate?). Answers to these types of 
questions can be used in two ways. First, the number of positive 
or negative comments can be counted. The more j^sitive comments, 
the better the progreun. Second, the comments can be classified 
into inherent categories: the content, the instruction, the 
facilities, the appropriateness of the objectives. Decisions 
about the appropriateness and worth of each category can be 
determined* In either case, the progreun's success is largely 
related to the perceptions of the participants who attended. 
Research Questions t 

From the literature, it seems evident that extensive 
resources are spent on communication training progrsuns and yet 
these progreuns are not always evaluated. Further, it is evident, 
as Laird (1985) suggests, that there are concrete methods of 
evaluating training programs. One type of organization that uses 
communication and communication-based training progreuns are 
hospitals. Many hospitals view themselves as a business — engaged 
in making profits and fighting off competition. The present 
study, therefore, was conducted to determine whether several local 
hospitals have communication training progreuns and wether the 
programs have been or are currently being evaluated. 
Specifically, the following research questions were used to guide 
this study s 



RQl: Have local hospitals ever uoed a communicauion training 

prograun (i.e., writing, interpersonal communication, 

public speaking, group leadership)? 
RQ2: If so, what is the background of the progreun (i.e., 

content, who initiated the progreun, where was it 

conducted, how long did it last)? 
RQ2: Why was it decided that the communication progrsun was 

necessary? 

RQ4: How successful was the program (i.e., formal 

evaluation, based on intuition or comments heard by 
trainer, pencil and paper test)? 

RQ5: WcS the communication training program conducted by 
in-house personnel or outside consultants? 

METHODS 

Subjects t 

For this study, training directors of five local hospitals 
(Allegheny General, Children's, Mercy, Presbyterian, and 
Shadyside)^ were interviewed.^ The directors were contacted 
by telephone and the study was explained to them. After the 
intentions of the study were made clear, the researcher asked the 
directors if he could interview them personally. Four of the 
directors agreed to meet face-to-face and one director, because of 
time constraints, asked to be interviewed over the telephone. 



^Appendix X contains the names of the training directors who 
were interviewed. 

2a training director is defined as a person who is responsible 
for creating, teaching, and/or evaluating the hospital's 
conmuni coition program(s) , regardless of whether that person is 
also a manager. 
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Inttr lanentatlon t 

An interview schedule was constructed by using the five 
research questions as guidelines. A copy of the actual 
questionnaire can be found in Appendix II. 
Procadures i 

During the first part of each interview, the researcher began 
by giving a brief background of what the study was about and why 
the it was being conducted. To quickly establish common ground, 
the researcher showed the training director a simple chart (Table 
1). 



Table 1 
Conmunication Training 
Academics Business 





* 


Theory 


* 


Theory? 


Content 


* 


Research 


* 


Research? 




* 


Training for 


* 


Training for 






future 




immediate future 




* 


Quizes, tests. 


* 


Prograun content 


Assessment 




written assign. 


* 


Job performance 




* 


Teacher 


* 


Trainer performance 






evaluation 







This chart was used to show that there seems to be some 
differences between academics and the business community in terms 
of the content and assessment of communication programs. It was 
explained to the directors that on the content dimension academics 
often concentrate on theory and research. The goal of training is 
mostly directed toward the distant future. In a business setting, 
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however, content may be based on theory and research, but training 
seems to be conducted for more immediate purposes. On the 
assessment dimension, academics often evaluate students by giving 
quizes, tests, and written assignments. Teachers, if evaluated at 
all, are usually assessed at the conclusion of the semester. 
Training programs in business contexts can be evaluated in a 
number of ways (e.g., whether the program content was correct, 
change in job performance, the trainer's performance, and so on). 
After this short model was made clear, it was explained to the 
director that the majority of questions or areas of discussion 
%^uld be related to these dimensions • 

During some interviews, the questions were asked in order* 
Other times, because some answers anticipated future questions, 
the interview schedule v^as rearranged. When ever an answer to a 
question or part of a question was not clear, the researcher asked 
more questions (probes) for clarification • When a director spoke 
of a specific program or a specific type of evaluation, the 
researcher asked if he could have a copy to study in detail* If 
any of these forms were part of a vendor's program, th;^ vendor was 
written and asked if they would provide the forms* After the 
interviews were completed, summaries of the conversation were 
recorded* Finally, within two days of the Interview, thank you 
letters were sent to each director expressing appreciation for 
time spent and information shared during the interview* 
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RESULTS 

Existence of Communication Training Programs t 

Two general types of coitonuni cation training progreuns were 
found to be used by the five hospitals interviewed: 1) some form 
of a "guest relations" program and 2) more specific and extensive 
communication topic areas. Taole 2 provides a summary of these 
findings • 

Table 2 

Types of Comsunication Training Programs 

Guest Interpersonal Public Group 

Hospital Relations Writing Communication Speaking Leadership 



ASH 


Yes 


Yes 


Yes 


Yes 


Yes 


Chlldr«n*8 


Yes 


7 


7 


7 


? 


Mercy 


* 


* 


* 


* 


* 


Presby 


Yes 


Yes 


Yes 


Yes 


Yes 


Shadyslde 


Yes 


No 


No 


No 


No 



Note: ? a Director not familiar with other departments 
* » Director newly hired — progreuns planned 



Table 2 shows that all hospitals have or are planning some form of 
guest relations program. The majority of these programs involve a 
discussion of basic communication skills # such as nonverbal 
communication, listening, and other interpersonaJ skills. The 
more detailed programs, such as writing, interpersonal 
communication, public speaking, and leadership are found at 
Allegheny General and Presbyterian-University hospitals (and, 
possibly, at Children's). 
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Background of thm Programs t 

Who Inltlatftd the programs ? The answers to this question 
were mixed. In some cases (e*g*. Children's) the director 
inherited a program that was already in existence • In other cases 
(e.g., AGH, Mercy, Preeby, ani Shadyside) , some parts or some 
programs had direct input from the training directors* In most 
cases, however, the programs had the support and approval by 
management* 

What did the programs involve? The guest relations prograuns, 
as mentioned above, usually briefly covered basic communication 
skills* Approximately two to three years ago (for most 
hospitals), the programs were inclement ed as part of a hospital- 
wide training program* Almost every employee was encouraged to 
attend these program* After the general training was completed, 
various follow-up sessions were implemented (e*g*, more workshops, 
newsletters, contests to pick f lonnel who were exceptional 
employees and used the skills discussed in the seminars, and 
various other activities)* The format of these training session 
often followed video tapt^s, lectures, and discussions* 

The more detailed communication programs contain information 
about specific topic areas* For example, Presbyterian-University 
Hospital provided such courses as "First Impressions,** **Person to 
Person Communication Skills for Handling Complaints," "Word 
Workout I," and "Wbrd Workout II" to name a few* An example of 
another program from Allegheny General Hospital concentrated on 
management (or leadership) training* The course covered a variety 
of interpersonal skills as well as provided examples of group 
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structure and how communication is related to these structures* 
For the most part, however, the content of these prograuns were 
protected by the directors. However, the directors did reveal 
that the majority of the sessions were led in a lecture and 
discussion format. Only Mercy Hospital was planning a management 
seminar that relied heavily on video tape. 

Where were the programs conducted ? In the majority of cases, 
the programs were conducted in-house. This means that either the 
hospital had special training facilities or that the trainer went 
to an on-site location. In only a few cases, seminars were 
conducted by outside consultants in outside facilities* These 
cases usually involved upper management. Apparently, some 
managers felt awkward about being trained by someone who had less 
status than they did. 

How long did the programs last? The programs ranged in 
length from two hours to four days. 
Decision to L:ipleBient Trainings 

General Responses : A variety of answers were given to this 
question, however, the overwhelming response was to make the 
hospital more profitable. Most directors explained that they, as 
well as management, viewed th^ hospital as a business. Therefore, 
the goal of training is to help make the business profitable, 
.whether by training employees to move into new positions, to treat 
customers more politely, or to help reduce cost. Presbyterian- 
University Hospital offered additional reasons such as 1) they are 
a teaching hospital and training in non-technical areas is not a 
foreign concept, 2) human resource has shown links between 
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training and increased per£onnance# and 3) by providing core 
curriculum for mauagement, the hospital is pir acting itself 
legalxv. 

Other Comments I In addition to these general comments # 
directors also mentioned that the guest relations programs were 
implemented as part of general training and orientation of new 
employees. As far as the more specific courses are concerned^ 
some employees voluntarily attended the programs while others were 
requested by management. An example of t' s later case would be 
if a particular department is having problems # they would call the 
training director and discuss the problem in detail. In some 
cases # the director would then establish a prograon designed to 
help solve the immediate departmental problems. 
Evaluation of Training Prr^graast 

Table 3 contains a summary of the different types of 
evaluation that the training directors used* 



Table 3 



Methods and Frequency of Evaluation 



Hospital 



Pencil and paper Some form of Interview or 
questionnaire following pre-test post- supervisor 
training session test design appraisal 



AOH 

Children's 
Meroy 
Presby 
Shadyside 



Always 
Always 

Alwayr. 
Always 
Always 



Possible 
Sometimes 



Sometimes 



Sometimes 
Sometimes 
Possible 



N6 
Always 
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Paper and pencil Questionnaire ! From Table 3 # It Is clear 
that every hospital Interviewed had some short of pencil and paper 
evaluation Immediately following either the guest relations or the 
more specific communication courses* The majority of these 
questionnaires were one page In length and typically asked 
questions about how well the material was prese£2ted, how clear the 
concepts were made during the lectures, and how well the 
Instructor presented the material. 

Pre-test/post-test designs i Also noted In Table 3 Is that 
Allegheny General and Presbyterian-University Hospitals also used 
a pre-test/post-test design with their training. For example, 
Allegheny General sends out a detailed registration form to 
managers enrolled In the Allegheny Leadership Development Program. 
This form specifically asks managers to list a set of ""performance 
objectives" or "projects" that they use In day-to-day practice, 
but may need to Improve. This format allows the managers assess 
their present skill levels before they attend the program. After 
the program Is completed, the managers iittempt to describe how 
well they met these performance objectives. Presbyterian- 
University Hospital, on the other hand, has developed short 
"quizes" for certain participants to complete before the program. 
Once the program is completed, then a similar quiz is given and 
participants are assessed based on the change that occurred 
between the first and second quiz. 

Job performance appraisal i In some cases # such as the 
programs offered at Presbyter ian-*University Hospital, there are no 
direct methods of assessing how well someone has applied the 



skills that were covered in a training session. In other cases# 
such as Allegheny General# Children's, and Shadyside Hospitals, 
more direct efforts of assessing job performance are attempted. 
For example, Allegheny General uses an appraisal form for its 
guest relations prograun that is completed by both the employee and 
the employee's immediate supervisor. The similarity and 
differences of these comparisons are discussed during appraisal 
review periods. Children's determines how well its employees are 
using the skills covered in the guest relations program by asking 
the parents of the sick children to complete a hospital 
satisfaction questionnaire. Each department is assessed on how 
well they treated the parents and patients. Management knows that 
the program is working when each department consistently receives 
good ratings. Finally, Shadyside has gone to .eat lengths to 
change every job description in the hospital and make specific 
requirements for each job and function. For example, a 
secretary's new job description might include a section on 1) how 
quickly to answer the telephone, 2) how to address the person 
calling, 3) how to be courteous to the caller, and 4) how to 
handle upset callers. Therefore, each individual is assessed on 
how well they have preformed specific communication skills. 
In-*hous0 Staff versus Outside Coasultantst 

Each hospital interviewed uses a combination of in-house 
staff and outside consultants to develop and teach their programs t 
In some cases, such as Children's and Shadyside, an outside 
consultant was hired to develop the program. After the program 
was started, in-house staff took over and ran the program. 
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Presbyterian hires consultants to teach certain courses when they 
have staffing problems. Allegheny General sometimes works with 
consulting firms to develop training tapes and other materials for 
in-house programs. Finally, Mercy has purchased a number of 
complete programs that are simply scheduled by in-house staff. 

DISCUSSION 

General conclusions » The results from this study demonstrate 
two encouraging points s communication training programs exist and 
the programs are evaluated in some way. First # every hospital has 
or is planning a communication-based training program. More 
importantly, however, specific content communication areas (e.g.# 
writing, public speaking, interpersonal communication, and 
leadership) are taught in at least two of the hospitals 
interviewed. It is clear, then, that these five hospitals view 
communication skills and communication training as an important 
part of thf^lr overall training programs. Second, every hospital 
that has cation-based training program has some method of 

evaluatinr thezie i^rograms, only if that method xs a short pencil 
and paper response immediately following each program. This 
fin^'ing both contradicts and confirms some of the arguments made 
in the literature. The assertion that training programs are 
rarely assessed does not fit with these findings. What fits is 
that the programs are rarely thoroughly evaluated through detailed 
quantification. Specifically, evaluation is usually of Laird's 
(1985) third types based on perceptions of trainees and their 
managers. In only some of the programs conducted at Allegheny 
General and Presbyterian do trainers attempt Laird *s second type 
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of evaluation: based on the achievement of learning objectives. 
Finally f in no cases that were discussed # are measurements used to 
establish baselines before and after training to determine how 
they affect the organization's overall goals. Therefore, it seems 
clear that much more investigation and research is needed to 
determine 1) if baselines can be constructed and 2) if training 
progrcuns can be accurately assessed using this type of 
evaluation. 

Limitations of the study : There are at least four 
limitations of this study that need to be noted. First # only five 
area hospitals were sampled. This population is too small to draw 
conclusive findings and generalize these findings to all 
hospitals . However, the hospitals that were interviewed do 
provide interesting points of reference that an additional # larger 
study could stem from. Second, because Mercy's staff was recently 
replaced # the training programs that they wish to use are not 
started yet. Therefore # the information given about these 
programs could change once the progreuns begin and the day-to-day 
problems begin to surface. Third# at times, there was resistance 
to sharing information about the content of the specific 
communication programs. Even though the researcher assured the 
directors that he taught, studies, and research much of the 
materials in these programs, some wez- reluctant to show sample 
handouts and so forth. Because of this resistance, it is not 
likely that any study attenipting to compare the content of 
business comnuni cation courses and academic communication 
courses could be conducted with much cooperation. However, 
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training directors were very willing to share evaluation forms and 
the like* Therefore, a future study concentrating on the 
evaluation process appears to be much more feasible* Finally, the 
last limitation that needs to be mentioned is that there were 
problems with scheduling. Because the training directors were 
extremely busy, finding time to discuss each Individual program 
was often difficult. Therefore, the directors usually choose to 
talk about one or two programs, while mentioning that they had 
several other programs as well. Future research projects shduld 
be limited to approximately 30 minutes of interview time. 
Although some interviews lasted up to two hours, 30 minutes would 
be plenty of time to ask several questions as well as not infringe 
on the training director's time. 

Future research t Evaluation will become more and more 
important as organisations continue to become more and more 
concerned with the cost of individual programs. It seems clear 
that decisions will have to be based on more than participants 
opinions of the course, content, and instructor. As this shift 
begins to occur, more than likely training directors will have to 
rely on more statistical techniques. With the present perception 
questionnaires., the majority of findings appear to be reported 
Ui^ing some sort of percentage. Percentages are a useful 
indicator # but may not be as useful as they could be. Additional 
breakdowns by some demographic variables (e.g., sex, age, 
department, years with organisation, and so on) might provide more 
information about what groups find the programs to be working best 
for them* In baseline studies, t-»tests and AHOVAs might provide 
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the statistical significance that is needed to say whf.iher a 
program was successful* In addition, significant le/ols of these 
test^ would also show how successful the progretin was — did it make a 
real change, did the organization get what it paid for? Studies 
proving or disproving that baselines are an accurate method of 
assessment seem particularly urgent* Is there a systematic method 
for determining and testing baselines? Are there other techniques 
in addition to determining baselines that would make evaluation 
more valid than its current state? Answers to these questions 
should provide the first step toward answering what appears to be 
a relatively simple question t if we buy your communication 
training program, how will we low that it's worth the money? 
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APPBHDIX I 



Evaluating Oommunication Consulting Questionnaire 



Ccnnpany Names 
Telephone Numbervs): 
Persons Contacted s 



1. Has your company ever used a communicav.ion training pro^rram? 
[Examples s writing, interpersonal communicfition, public speaking, 
group leadership?] 



2. Can you give me a little background on the program? 
[Examples s who initiated the program, what did it involve, where 
%AS it conducted, how long did it last, who conducted it, etc...] 



3. Why did you or someone else in your organization decide that 
communication training was needed? [Examples s to improve 
productivity, in^rove morale, part of standard training provided 
to employees, etc . . . ] 



4. How successful was the program? [Exeunpless did you have a 
formal evaluation procedure, was it based on intuition or comments 
you heard, pencil and paper test, job performance ai^raisal, 
etc.] 



5. Were the communication training progreun conducted by in-house 
personnel or by outside consultants? 



APPENDIX II 



Training Directors Interviewed 



Sam Mentser 

Allegheny General Hospital 
Coordinator Educational Services 
Management and Organizational Development 



Joan K. Shames 

Children's Hospital of Pittsburgh 
Patient Representative 



William F« Wilson 

Mercy Hospital 
Director 

Organisational Development 



Hoikica J« Joyce 

Presbyterian-University Hospital 

of Pittsburgh 
Training Specialist 



Rachel Cupcheck 

Shadyside Hospital 

Bnployee Relations Coordinator 



